The present invention relates to textual information contained in video signals, and in particular to the capture and searching of such information to identify desired video programming.
The consumer electronics industry has created many stand alone products for specific functions, such as television viewing, video recording and playback, broadband video receivers, playing recorded music and broadcast music. Some devices combine functions, such as the combination TV/VCR, and the audio cassette/AM/FM receiver to name a couple. One direction that consumers are moving is toward larger televisions located within a family room or living room with accompanying high quality stereo. Multiple components are required, each providing separate functions. A large screen television based on a 19 inch to 40 inch picture tube, or 46 inch to 60 inch projection system is used and viewed from a distance of two to five meters. The television includes a tuner for receiving and decoding National Television Systems Committee (NTSC) signals, infrared receiver circuitry for a remote control, and in many cases stereo and surround sound integrated into it, making it a very expensive device. Further, the consumer likely has a video recorder/player, and perhaps a cable box/set top box to receive cable or satellite transmission which may also include a video tuner and other electronics to handle modulated, compressed and encrypted video signals. In addition, a consumer is also likely to have a separate stereo system complete with CD player, tuner and other audio attachments, such as speakers. This duplicates much of the functionality of the television system and adds to the cost of a home entertainment center.
Most consumer electronic devices come with remote control devices, which as in the case of the television above, require sensing circuitry to receive and process the signals from the remotes. Such controls are typically based on IR signals which can be interrupted by someone walking in front of it, and are not able to be used in a different room from the receiver. While a few have on-screen programming functions, and there are special remote control devices which can be programmed to control multiple consumer electronic devices found in a home entertainment center, there is little consistency between such controls. In addition, there is no good way to use different remote control units to control different programs generating the information displayed in various windows on the screen. This leads to confusion of the consumer, and the classic case of the blinking xe2x80x9c12:00xe2x80x9d as consumers become frustrated trying to master all the protocols required to appropriately control their electronic devices. Trying to program a VCR to record a program in the future can also be quite difficult. The expense of the additional circuitry in all the devices to accomplish these functions is borne by the consumer.
Multimedia based personal computers today are configured with CD Rom drives, and speakers as well as graphics drivers for displaying graphics on a monitor attached to the computer. CD Rom drives are capable of both reading data, such as computer programs, and reading audio information such as music which is output from the attached speakers. Multimedia titles for running on a PC usually are distributed on CD Rom, and involve such things as animated encyclopedias and other books, as well as games that may incorporate video clips which can be shown on a PC display. More and more of the video information on such CDs is compressed in accordance with Motion Picture Experts Group (MPEG) standards and requires commercially available software or circuitry to decompress it and process it for display. The display signal is typically of VGA quality. Some add on products for PCs even provide a connection to video feeds from multiple sources for playing in a window on the monitor screen. Typically, the computer has a video graphics adapter (VGA or SVGA) card which processes all the information to be displayed on a monitor and the monitor itself is basically a picture tube that shows only what it is sent with very little processing. However, most PC displays are small, and not suitable for viewing by multiple people at the same time. It often happens that when a family gets a new program such as a game, animated book or educational game, everyone wants to see it being used for the first time, and they huddle around a small display and vie for positions. In addition, there is no good way for multiple users to interact on a single computer. For viewing video feeds from cable or satellite, large screen consumer television sets are most commonly used in the home entertainment center.
As can be seen, there is great duplication of function between the consumer electronics and personal computers, both of which are more and more likely to be found in a family room or great room of a home. This duplication of function leads to much more money being spent to fully outfit the home entertainment center and provide additional functions. When one component fails, since it has duplicate function, it is expensive to replace. One system that tried to solve some of the above problems is shown in U.S. Pat. No. 5,192,999 to Graczyk et al. That system has a television circuit and an audio circuit within a personal computer, both of which are controlled by a remote control device. The television circuit is used to receive common cable or broadcast video signals in NTSC format, which inherently have a lower quality than video signals currently broadcast by digital broadcasting satellite. NTSC format signals are interlaced, which means that every other line is refreshed during each scan of the picture tube. For example, odd lines would be refreshed during a first scan, and even lines during a second scan. Since there are 60 scans per second, odd lines are refreshed 30 times per second, and even lines are refreshed 30 times per second. VGA monitors refresh at least 60 times per second, providing a more coherent spacial and temporal image. The NTSC signal is a lower bandwidth signal than digital MPEG, which has a much higher bandwidth and allows productive use of even higher resolution monitors. While Graczyk et al. does convert the NTSC signals to VGA format for display by a data quality analog monitor, the signal quality is limited because of the lower bandwidth transmission. NTSC signals have a great effect on the type of text that can be displayed. With an interlaced display, the text displayed in normal fonts appears to jump as alternate lines are refreshed. There have been several attempts to design fonts that minimize this jump effect, but none have worked well. A VGA display, refreshing each line with every scan of the tube, does not have this problem, and provides a much sharper and readable image for text.
The monitor described by Graczyk is shown as a standard PC analog VGA monitor, and reference is commonly made to a single user. Such monitors are fairly small, having a maximum viewing area of 17.5 inches (44.5 cm) at the high end of the PC market. They are not nearly suitable for viewing in a home entertainment environment. They are designed for close viewing, having pixels very close together. Current home entertainment systems are much larger and expensive due to all the other circuitry they have as described above.
One satellite broadcast system is that provided by DirecTV, a unit of GM Hughes Electronics. Direct Broadcast Satellites, xe2x80x9cDBSxe2x80x9d provide more than 150 channels of high-quality MPEG based video, sound and data to 18 inch (45.7 cm) receiving antennas. Rights to make subscriber terminals are licensed by DirectTV, and several other companies contributing technology. News Datacom Corporation provides encryption and security for the DBS system, providing decryption keys, software and an access card for each subscriber terminal. Thomson Consumer Electronics provides consumer subscriber terminals in the form of set top boxes called integrated receiver/decoders xe2x80x9cIRDSxe2x80x9d which convert the high-quality television signals into NTSC for showing on a standard television. Each IRD demodulates, decodes, decrypts and outputs video and data, which is displayed on a subscriber television. Error correction, decompression and demultiplexing to separate out video data from other data is also provided in the IRD. While S Video output is provided, there is no provision for monitor/VGA quality output capability. Other sources of MPEG based video include optical fiber based cable systems, compact disk, video clips available on the Internet network, both old and proposed HDCD (high-definition compact disk) formats, and other existing and proposed satellite, digital cable, and asynchronous-transfer mode (xe2x80x9cATMxe2x80x9d)-based systems as well as wireless digital broadcasts.
A multipurpose computer system is provided with circuitry to control consumer electronics, such as a large monitor or television for group interaction and of text, graphics and video in a home entertainment environment. The circuitry provides audio and video tuning capability for display of received high quality video signals on the monitor without an intermediate conversion to a lower quality NTSC format. The circuitry decodes the received video signals and converts them to VGA format which provides images of higher quality than NTSC. The monitor is capable of directly displaying VGA signals, and is not required to have a television tuner circuit, remote control, or audio circuitry. This significantly reduces the cost of the monitor and improves the quality of display as opposed to typical large screen televisions which contain complex tuning circuitry.
In one embodiment, the high quality video signals are received from satellites broadcasting digital video signals, digital cable signals and other wireless digital broadcasts. One example is direct broadcasting satellite xe2x80x9cDBSxe2x80x9d signals having a quality of video signal higher than that provided by NTSC broadcasts. The DBS signals are MPEG-compressed video, audio and data. The DBS signals are received by a video receiver circuit board which is compatible with a standard personal computer peripheral component interconnect (PCI) bus and fits within the chassis of a personal computer. The video receiver cards have panel connectors for receiving coax cable from an antenna which directly receives the DBS signals. The card has functional blocks comprising a satellite tuner, digital demodulator, forward error correction, conditional access and decryption/demultiplexing. The demultiplexor provides the capability of receiving data packets which range from information on television programs being broadcast, to computer programs for downloading into computer memory.
The video receiver card outputs digital video, audio and data streams onto the peripheral component interconnect xe2x80x9cPCIxe2x80x9d bus where it can be accessed by the computer main processor for manipulation and storage. When stored in a personal computer memory, the program information may be organized in a commercially available database format. This permits the use of database functions to be applied to the data. Rather than being stuck viewing the program information as provided by the broadcaster, the data immediately becomes manipulable by database commands, allowing queries of program information. Searches are used to show programs having certain actors or subject matter, and even have programs meeting the query automatically recorded. One use entails a user designating a series to be recorded, and the database keeps track of what has been recorded or already viewed, avoiding duplicate recording, as well as providing easy access to the recorded programs. In general, by capturing data along with video and audio in a form which a personal computer can process, the possibilities for computer applications are endless. Interaction with movies, classrooms, other players of games such as golf and a host of other possibilities becomes clear.
Further embodiments of the system include a settop box version, where all the circuitry is integrated into one or two cards in a box designed to sit on top of a television set having VGA input. In another version, all the circuitry is included inside of the television chassis.
In one embodiment of the invention, closed caption information provided in video signals is captured, and stored in the database for searching. This provides the ability to instantly find current programming discussing events that a user is interested in. In a further embodiment, speech recognition circuitry is used to convert speech to text or commands for similar searching capability. Given current speech recognition capabilities, not all words may be recognized. Only those that are recognized are stored in the database or used as a command. Much of the information so captured is not relevant in a certain amount of time, so a data retention mechanism is used to identify old data and delete it from the database once the database has exceeded its allocated resource level. A standard FIFO algorithm based on the time expired since the programs described have been shown is employed. Further algorithms are user selectable to relevancy rank data in accordance with user preference.
A video graphics adapter (VGA) card also coupled to the PCI bus converts the digital data stream into VGA and video signals for display on the monitor. In addition to being a home entertainment system, a fully functional computer system forms the heart of the entertainment system. Instead of buying all the individual consumer electronics parts, such as a large screen television, settop box for receiving broadband video, audio amplifier/receiver, CD player, universal remote control, video game machine, answering machine and fax, and a personal computer as some consumers do today, they need only buy the home entertainment system of the present invention. The cost is about the same as that for the individual traditional consumer electronics parts, making the personal computer essentially free.
Support for remote control of both the personal computer and the monitor functions is also provided in the personal computer as well as standard PC VGA graphic display functions normally associated with personal computers. Additional functions also become available based on the integration of devices and digitized data, video and audio. In this manner, no audio, remote control or channel tuning electronics is required in the display, which results in a much lower price for a large monitor suitable for home entertainment. Both a handheld remote control having standard television and in one embodiment, video cassette recorder controls, and a full function remote keyboard having similar standard television and video cassette recorder controls are provided. Both transmit key signals identifying the key pressed, as well as a signal identifying the source of the key signal. The key signals are preferably RF signals typically in the megahertz range, but can also be IR or other suitable form of radiation. RF signals have the advantage over infrared xe2x80x9cIRxe2x80x9d signals in that they are not interrupted by someone walking between the remote and the receiver. Light source interference and jamming of other IR receivers is minimized by using RF signals. It may also be used in a different room from the receiver.
The personal computer contains suitable receiving circuitry, which provides indications of the keys being pressed, with the identity of the remote control device sending the signal. In one embodiment, an industry standard architecture (ISA) board or module is plugged into both the keyboard and mouse ports. Another bus, such as a serial, RS232 or microchannel bus may also be used. The board contains RF receiver circuitry which receives the RF signals, decodes them and routes them to the appropriate port for processing. A receiving module is programmed to distinguish between the sources of the remote control transmitted signals, and directs keystrokes to the keyboard port, and mouse movements to the mouse port. The signals at the two ports then control which program the remote key signals affect. If the handheld remote is activated, the key signals usually would control what channel is being displayed, or cause a recorded program to fast forward, play or reverse. It could however be selected to move to the next image or otherwise control a program designed to display pictures taken on a digital camera. Keyboard keys are more likely to control a computer program application such as a word processor, spread sheet or electronic mail program. In one embodiment, the television like controls are used to override the remote hand held television controls in channel selection. Just as easily, the hand held remote controls paging through electronic mail. Additionally, the hand held remote has a pointing device, such as a trackball, or miniature joystick with normally associated keys for selecting functions identified by the cursor on the display being moved by the pointing device. The keyboard contains an integrated touchpad for performing similar function, and additionally for transmitting signatures, providing the ability to ensure that an authorized user is requesting a transaction. Given the entertainment room environment, a touchpad provides the further benefit of being able to withstand greasy popcorn and chicken fingers which might foul other pointing devices. In a further embodiment, each remote control device has its own unique cursor associated with it. In this manner, it is easier for the user of a particular device to know which program will be affected when the remote is used.
The remote controls are also used to control the data supplied to more than one display. In one embodiment, a display is located in a den, where it is displaying a financial application, such as a checkbook balancing program. The personal computer is set up to drive multiple monitors, and the remote control keyboard is assigned to the monitor in the den to control the running and display of information from the financial application.
The remote control devices are used to control one or more cursors displayed on the monitor as part of a graphical user interface into the home entertainment system. Cells displayed as boxes containing descriptive text are associated with video programs and with video programs and with computer programs, may be selected to view and/or run the programs. Several icons, such as pictures, video clips or list boxes indicating functions, like record are used to provide functions by dragging and dropping cells onto the icons. Other icons include remind, buy, picture in picture and view full screen to name a few. In one embodiment, each remote control controls its own cursor to allow multiple users to control different portions of the system.
In a further preferred embodiment, the remote control devices are equipped with a user mechanism to reduce the power of the transmitted radio frequency signals depending on their distance from the computer. Circuitry is provided to detect the power level of the signal received and provide an indication on the computer screen, or transmit power level signals back to the remote control device indicative of the power level of the signal received. The remote control device has means to adjust the power, either by the user varying the resistance of the power amplifier, or by digital adjustment of the power amplifier based on the power level signals transmitted from the computer. The ability to control the power level of the signals helps lengthen the time required between replacing batteries which power the remotes.
In yet a further embodiment, remote earphones are provided. The personal computer has the capability to generate separate sound tracks for each window being displayed on the monitor. It assigns each to one or more sets of earphones and transmits a local FM signal or broadcast IR or RF which is received and played by the earphones set to the right channel. The earphones have tuning circuitry that allows them to be set to a desired channel, or they may be fixed to different frequencies, and software is used to set the frequency or frequencies on which the audio associated with a window is broadcast.